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Renyi  (1959)  defined  the  maximal  correlation  )p  between  a  pair 
of  random  variables  (U,  W)  as  0  r. . !  r.  •  . 


sup { :  V(f(U))  >  0,U(ff(TU))  >  0}, 
WVU[U))V{g{W))  m  h 

where  the  supremum  is  taken  over  all  functions  of  U  and  W  with 
finite  second  moments.  In  this  paper  we  derive  the  asymptotic  dis¬ 
tribution  of  the  estimate  of  the  Renyi  correlation  coefficient  based 
on  a  sample  of  independent  observations  under  the  assumption  that 
(U,W)  axe  independent  and  assume  only  a  finite  number  of  values. 


1.  INTRODUCTION 


l-r 


Renyi  (1959)  defined  the  maximal  correlation  p  between  a  pair  of  random 
variables  ( U,W )  as 


sup{ 


an>U(U).}(W)) 

vnnu))v(g(w)) 


:  V(/([/))  >  0,V(g(W))  >  0}, 


where  the  supremum  is  taken  over  all  functions  of  U  and  W  with  finite  second 
moments.  One  of  the  attractive  features  of  the  Renyi  maximal  correlation  is 
that  U  and  W  are  independent  if  and  only  if  p  =  0. 

An  explicit  evaluation  of  the  Renyi  maximal,  correlation  is  not  available 
for  a  general  random  variable  (U,  W)  except  in  very  special  cases.  A  case 
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of  special  interest  is  that  of  the  bivariate  normal  distribution.  The  Renyi 
maximal  correlation  for  a  bivariate  normal  distribution  with  correlation  r  is 
Jr |,  testifying  to  the  fact  that  r  =  0  implies  independence.  We  will  now  give 
a  direct  evaluation  of  the  Renyi  maximal  correlation  p  when  U  and  W  take 
only  finite  number  of  values. 

Suppose  that  U  takes  on  only  a  finite  number  of  values  aj , . . . .  or+i  and 
W  takes  on  only  a  finite  number  of  values  To  avoid  trivialities, 

we  will  assume  that 

P(U  =  a,-)  >  0,  for  1  <  i  <  r  +  1, 

P(W  =  f3j)  >  0,  for  1  <  j  <5  +  1,  and 

r  >  s.  (1) 


In  this  case  we  can  replace  the  bivariate  random  variable  ( U,W )  by  Z 

{X, Y}  a=  (Xu...,Xr,Yu...,Ys)'  where  A',  =  I{U  =  a*),  1  <  *  <  r  and 
Yj  =  I(W  =  f3j),  l  <  j  <  s,  and  where  /(•)  stands  for  the  indicator  function. 


In  the  rest  of  this  paper,  we  use  the  expression  U  and  W  take  on  only  a 
finite  number  of  values  to  mean  what  we  have  said  above,  including  assump¬ 
tion  (1). 


It  is  clear  that  (Ari, . . . ,  Xr,  Yi,...,Ys)'  is  a  one-to-one  function  of 
( U ,  W)  and  thus  for  all  statistical  purposes,  the  random  variable  (17,  W)  can 
be  replaced  by  {X,Y}.  Notice  that  in  view  of  (1),  A’i,...,Xr  are  linearly 
independent  and  A’r+i  is  a  simple  linear  function  of  A\ , . . . ,  Afr.  Similarly 
Y\,...,Y3  are  linearly  independent  and  17,4-1  is  a  simple  linear  function  of 
Yi, . . . ,  Ya.  We  can  give  an  easy  explicit  form  for  the  Renyi  maximal  correla¬ 
tion  of  (U,  W)  in  terms  of  the  variance  covariance  matrix  of  {X,  Y}.  To  do 
this  we  need  to  set  up  the  following  definitions. 


Let  E(X)  =  7.  E( Y)  =  6, E((X  - 7)(X  -  7)')  =  T,  E((Y  —  S)( Y  -«)')  = 
A,  and  E((X.  —  7)(Y  —  6)')  =  0.  Then  T  and  A  are  the  variance  covariance 
matrices  of  X  and  Y,  respectively  and  0  is  the  covariance  matrix  between 
X  and  Y.  We  can  rephrase  (1)  by  saying  that  F  and  A  are  of  full  rank. 

Notice  that  the  most  general  function  f(U)  of  U  is  no  more  than  a  linear 
function  a'X  of  X  for  some  vector  a.  Similarly,  the  most  general  function 
g(W)  of  W  can  be  replaced  by  a  linear  function  b'Y  of  Y  for  some  vector 
b.  Thus  the  maximal  Renyi  correlation  p  is  given  by 

p  =  sup{a'0b/ \/(aTa)  (b'Ab)  :  a'Fa  >  0, b' Ab  >  0}.  (2) 
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Let  M  and  N  be  nonsingular  matrices  such  that  M'M  =  T  and  N'N  =  A. 
We  can  simplify  (2)  to  read  as 

p  =  sup{a'(M')~10iV-1b  :  a'a  =  l,b'b  =  l}.  (3) 

From  standard  matrix  manipulations,  the  maximization  problem  in  (3)  can 
be  solved  and  we  find  that 

P  =  y/m 

where  p\  =  maximum  eigenvalue  of  (N')~1QI . 

We  may  note  that  is  the  first  canonical  correlation  between  X  and 
Y,  as  defined  in  the  literature  (e.  g.  Anderson  (195S),  p.  295  ).  Canonical 
correlations  can  be  defined  for  any  two  random  vectors  X  and  Y  which  need 
not  consist  just  of  indicator  random  variables  as  considered  here. 


2.  ESTIMATION  OF  THE  RENYI  MAXIMAL  CORRELATION 

BASED  ON  A  SAMPLE 

Suppose  that  we  have  a  sample  {(Ut,Wt),  1  <  t  <  n)  of  independent 
and  identically  distributed  observations  on  (17,  W).  How  should  we  estimate 
p  and  what  will  be  the  asymptotic  distribution  of  this  estimate?  We  propose 
to  address  these  questions  in  this  paper. 

This  problem  does  not  seem  to  have  an  easy  solution  when  (17,  W)  is 
a  general  bivariate  random  variable.  However  when  U  and  W  take  on  only 
a  finite  number  of  values  as  described  in  Section  1  and  p  =  0,  we  are  able 
to  give  a  solution  to  the  questions  posed  above.  The  final  result  is  given  in 
Theorem  4  of  Section  3.  We  announced  this  result  in  Sethuraman  (1977). 

We  can  replace  (l\,  Wl (17n,  Wn)  by  (Zx,...,Zn)  =  (Xt,Y<),  1  < 
t  <  n ,  where  Xti  —  I{Ut  =  =  I{Wt  =  fij),  1  <  t  <  n,  1  <  i  <  r,  1  < 

j  <  s,  by  using  the  method  described  in  Section  1. 

Let 

A‘,  =(l/n)  V  XU, 

\<t<n 

r<  =  (l/n)  E 

1  'v/  n!  11 
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Cii'  —  l^/^O  ^  ^  Xj -Xj< , 

1  <t  <n 

da-  =  (1/n)  Y"  YtiYn'  -  YY-,  and 

l<(<n 

c<i  =  (1/n)  £  XuYtv-XiYi'. 

l<t<n 


The  matrix  (  ^  ) 

(Xi,  Yi), . . . ,  (X„,  Yn 


represents  the  sample  variance  covariance  matrix  of 
)• 


More  generally,  for  a  =  (a1,...,ar')'  £  Rr  and  b  =  (b\, . . .  ,b3)'  £  R$ , 
define  __ 

Jf(a)  =  (l/n)^(^ajA'-i), 

t  i 

r-(b)  =  (i/n)^(^tJrI;), 

<  i 

c{  a,  a)  =  (1/n)  a«X'tt)2  ~  X'(a)2, 

t  i 

d( b,  b)  =  (1/n)  £(£  <>yr,;)2  -  ?(b)2, 

t  i 

e(a,b)  =  (l/n)£(5>,X,.)(X>V<j)  “  X(a)?(b),  and 

t  i  j 

r(a  b)  =  i  e(a’  b)/ \/c(aia)  d(b,  b)  if  the  denominator  ^  0 
l  0  if  the  denominator  =  0  . 

That  is  r(a, b)  is  the  sample  estimate  of  corr(JT  bjYj).  Let 

r*  =  supr(a,  b). 

a,b 


Then  r*  is  the  sample  maximal  linear  correlation  between  X  and  Y.  It  is 
also  the  sample  Renyi  maximal  correlation  based  on  (17;,  W\)  . .  . ,  (Un ,  Wn). 
It  is  natural  to  use  r*  as  an  estimate  of  p. 


3.  THE  ASYMPTOTIC  DISTRIBUTION  OF 
THE  SAMPLE  RENYI  CORRELATION  COEFFICIENT 

We  continue  to  make  the  assumption  that  U  and  W  take  on  only  a  fi¬ 
nite  number  of  values.  Throughout  this  section  we  will  make  the  additional 
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assumption  that  p  =  0.  Under  these  assumptions,  we  will  obtain  the  asymp¬ 
totic  distribution  of  r*  in  Theorem  4.  Before  proving  this  theorem  we  will 
establish  some  preliminary  results. 

Theorem  1.  Let 


y/nEy  def  ,  , 

£)  )  {sk,k')i 


1  <  k ,  k1  <  r  +  s. 


Then  S 


E  in  distribution,  where  S 


and  the  elements 


{£,j}  of  E,  1  <  i  <  r,  1  <  j  <  s,  have  a  multivariate  normal  distribution 
with  mean  0  and  cou(£,j, =  r*,<  Ajj<,  1  <  i,i'  <  r,  1  <  j,j'  <  s. 
Proof.  Notice  that  all  moments  of  X  and  Y  are  finite.  Furthermore, 
cov(Xi,Xi> )  =  T and  cov(Yj,Yj>  )  —  This  implies  that 

Eii'  ^  r,„<,  Djj'  >  Xj j ' 


in  distribution  (and  also  w.p.  1),  1  <  i,i'  <  r,  1  <  j.j1  <  s.  Again,  since 
p  =  0,  it  follows  that  cov(X{Yj,  Xi'Yj')  =  Tu'Ajj'.  Furthermore,  y/neij  is 
the  normalized  sample  mean  of  X{Yj.  From  the  multivariate  central  limit 
theorem,  it  follows  that  y/nE  =  {y/neij,  1  <  i  <  r,  converges  in 

distribution  to  E  which  has  a  multivariate  normal  distribution  with  means 
0  and  with  cov(£ij,  £i'j')  =  Tu'Ajj>,  1  <  z,  i'  <  r,  1  <  j,  j'  <  s.  This 
completes  the  proof  of  the  theorem.  0 


Note  that  the  joint  distribution  of  E  above  can  be  stated  more  concisely 
as  follows,  using  the  vec  and  Kronecker  product  notations  for  matrices.  The 
distribution  of  vec  E  is  multivariate  normal  with  mean  0  and  covariance 
matrix  r  (g>  A. 


Theorem  2.  The  limiting  distribution  of  y/nr*  is  the  distribution  of 

sup{a'Eb/\/(aTa)  (b'Ab)  :  aTa  >  0,b'Ab  >  0}, 

where  E  has  the  distribution  specified  in  Theorem  1. 

C  E 

Proof.  Let  S  =  {  all  matrices  of  the  type  (^,  ^  )  where  C  and  D  are 

positive  definite  matrices  } .  Let  the  function  /  on  S  be  defined  as  follows: 

f{E>  D)=Vnr 


=  sup  y/nr(  a,  b) 

a,b 

y/na'  Eh 

=  sup  -;===  . 

a,b:a'Ca>0,b'Db>0  y(a'Ca)  (b'Db) 
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It  is  easy  to  see  that  if  Sk  is  any  sequence  of  nonnegative  definite  matrices 

r  e 

such  that  Sk  — >  £  =  (_ ,  “  )  pointvvise,  then  f(Sk)  — *•  /(E).  Thus  from 

the  invariance  principle  for  functions  of  a  convergent  sequence  of  random 
variables,  it  follows  that  the  limiting  distribution  of  y/nr*  is  the  distribution 
of 

sup{a'Eb/-\/(aTa)  (b'Ab)  :  a'Ta  >  0,b'Ab  >  0}.  0 

Theorem  3.  Let  Erxs  be  a  random  matrix  such  that 

vecE  ~  MAr(0,T  ®  A). 

Let  M'M  =  T,  and  N'N  =  A,  where  M,  N  are  square  matrices  of  order  r 
and  s  and  of  full  ranks.  Then 

vec((M')-1EiV-1)  -  MN(0,Ir®I,), 

and  the  distribution  of 

(N'y'z'M-'iM'y'zN-1 

is  the  Wishart  distribution  W(Is,r). 

Proof.  This  is  easily  proved  by  direct  computation  from  one  of  the  standard 
definitions  of  the  Wishart  distribution.  See  Anderson  (1958),  p.  157.  0 

Theorem  4.  The  limiting  distribution  of  y/nr*  is  the  distribution  of 
vA7  where  Ai  is  the  maximum  eigenvalue  of  W,  where  W  has  a  Wishart 
distribution  W(I„,r). 

Proof.  From  Theorem  2,  the  limiting  distribution  of  y/nr*  is  the  distribu¬ 
tion  of  _ 

sup  {a'  Eb/v/aTa)  (b'Ab)  :  aTa  >  0,b'Ab  >  0}. 

Let  M'M  =  T,  and  N'N  =  A,  where  M,N  are  square  matrices  of  order 
r  and  s  and  of  full  ranks  as  in  Theorem  3.  Then 

sup{a'Eb/ /(aTa)  (b'Ab)  :  a'Ta  >  0,  b'Ab  >  0} 

=  sup{a'(M')-1ElV~1b/\/(a'a)  (b'b)  :  a'a  >  0,  b'b  >  0} 

=  sup{a'(M')-1EAr~1b  :  a'a  =  l,b'b  =  l} 

=  /max  eigenvalue  of  (N')~1'E'M~1(M')~1'E.N~l . 

Now,  from  Theorem  3,  (N')~1E'M~1(M')~1EN~1  has  a  Wishart  distribu¬ 
tion  W(Is,r).  Thus  the  limiting  distribution  of  y/nr*  is  the  distribution  of 
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y/X\  where  Ai  is  the  maximum  eigenvalue  of  W  where  W  has  a  Wishart 
distribution  W(ISlr).  0 


Remark.  Theorem  4  also  establishes  the  asymptotic  distribution  of  the 
sample  canonical  correlation  coefficient  based  on  two  random  vectors  X  and 
Y  for  which  corr(a'X,  b'Y)  =  0  for  all  vectors  a  and  b.  Notice  that  we  did 
not  have  to  assume  that  X  and  Y  have  multivariate  normal  distributions. 
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